Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA.

With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher's question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.

Ethics and administrative jurisprudence of the rights of subjects of research in pandemic (COVID-19): role of the scientific ethics committees: Chile / Ética y jurisprudencia administrativa de los derechos de los sujetos de investigación en pandemia (COVID-19): función de los comités éticos científicos: Chile

RESUMEN: Todo gobierno debe reaccionar rápida y efectivamente ante cualquier pandemia, Chile no es la excepción y apoyado en el estado de Excepción Constitucional, ha tenido que implementar medidas que podrían involucrar poca información sobre las percepciones de las personas y las reacciones durante la implementación de las restricciones. Las instituciones internacionales de salud han determinado que es un deber moral realizar investigaciones que generen evidencia que promuevan y mejoren la atención de la salud y la mitigación de la pandemia, instando a reducir los "obstáculos" prácticos de la revisión ética. Los objetivos de este trabajo fueron analizar desde las perspectivas de las consideraciones éticas y jurídicas, el rol que cumplen los Comités Éticos Científicos en el manejo y la protección de las personas durante la pandemia de la COVID-19. La metodología de trabajo se basó en la recolección de la información de Instituciones nacionales e internacionales de Salud y luego analizarla según la jurisprudencia administrativa del gobierno de Chile. Se concluye que los cambios de criterios que deben observar los CECs en el proceso de revisión de los protocolos de los proyectos de investigación científica, deben velar por proteger los derechos de los pacientes y sujetos de investigación en cuanto puede involucrar información sensible, más aún, si se consideran las graves consecuencias de su transgresión, dar un sentido distinto al que corresponda a las normas sobre derechos de pacientes, puede resultar en "falta de servicio" y eventual vulneración en los derechos del sujeto de investigación. La labor de los CEC, debe realizarse siempre desde una interpretación restrictiva, reconociendo la función pública que cumplen como parte integrante de la labor ética encomendada por el legislador al efecto.

SUMMARY: Every government must react quickly and effectively to any pandemic, Chile is no exception and supported by the state of Constitutional Exception, it has had to implement measures that could involve little information about people's perceptions and reactions during the implementation of the restrictions. International health institutions have determined that it is a moral duty to carry out research that generates evidence that promotes and improves health care and the mitigation of the pandemic, urging to reduce the practical "obstacles" to ethical review. The objective of this study was to analyze from the perspectives of ethical and legal considerations, the role that Scientific Ethics Committees play in the management and protection of people during the COVID-19 pandemic. The methodology used was based on collecting information from national and international Health Institutions and then analyzing it according to the administrative jurisprudence of the Chilean government. It is concluded that the changes in criteria that the CECs must observe in the process of reviewing the protocols of scientific research projects, must ensure the protection of the rights of patients and research subjects insofar as it may involve sensitive information, even more if the serious consequences of its transgression are considered. Giving a different meaning to the one that corresponds may result in "lack of service" and eventual violation of the rights of the research subject. The task of the CEC, must always be carried out from a restrictive interpretation, recognizing the public function that they fulfill as an integral part of the ethical work entrusted by the legislators to that effect.

Evaluation of electronic recruitment efforts of primary care providers as research subjects during the COVID-19 pandemic.

BACKGROUND: Recruiting healthcare providers as research subjects often rely on in-person recruitment strategies. Little is known about recruiting provider participants via electronic recruitment methods. In this study, conducted during the COVID-19 pandemic, we describe and evaluate a primarily electronic approach to recruiting primary care providers (PCPs) as subjects in a pragmatic randomized controlled trial (RCT) of a decision support intervention. METHODS: We adapted an existing framework for healthcare provider research recruitment, employing an electronic consent form and a mix of brief synchronous video presentations, email, and phone calls to recruit PCPs into the RCT. To evaluate the success of each electronic strategy, we estimated the number of consented PCPs associated with each strategy, the number of days to recruit each PCP and recruitment costs. RESULTS: We recruited 45 of 63 eligible PCPs practicing at ten primary care clinic locations over 55 days. On average, it took 17 business days to recruit a PCP (range 0-48) and required three attempts (range 1-7). Email communication from the clinic leaders led to the most successful recruitments, followed by brief synchronous video presentations at regularly scheduled clinic meetings. We spent approximately $89 per recruited PCP. We faced challenges of low email responsiveness and limited opportunities to forge relationships. CONCLUSION: PCPs can be efficiently recruited at low costs as research subjects using primarily electronic communications, even during a time of high workload and stress. Electronic peer leader outreach and synchronous video presentations may be particularly useful recruitment strategies. TRIAL REGISTRATION: ClinicalTrials.gov , NCT04295135 . Registered 04 March 2020.

Recruitment and retention of participants in longitudinal studies after a natural disaster.

Climate change and population growth will increase vulnerability to natural and human-made disasters or pandemics. Longitudinal research studies may be adversely impacted by a lack of access to study resources, inability to travel around the urban environment, reluctance of sample members to attend appointments, sample members moving residence and potentially also the destruction of research facilities. One of the key advantages of longitudinal research is the ability to assess associations between exposures and outcomes by limiting the influence of sample selection bias. However, ensuring the validity and reliability of findings in longitudinal research requires the recruitment and retention of respondents who are willing and able to be repeatedly assessed over an extended period of time. This study examined recruitment and retention strategies of 11 longitudinal cohort studies operating during the Christchurch, New Zealand earthquake sequence which began in September 2010, including staff perceptions of the major impediments to study operations during/after the earthquakes and respondents' barriers to participation. Successful strategies to assist recruitment and retention after a natural disaster are discussed. With the current COVID-19 pandemic, longitudinal studies are potentially encountering some of the issues highlighted in this paper including: closure of facilities, restricted movement of research staff and sample members, and reluctance of sample members to attend appointments. It is possible that suggestions in this paper may be implemented so that longitudinal studies can protect the operation of their research programmes.

Metastatic gastroesophageal cancer in older patients - is this patient cohort represented in clinical trials?

BACKGROUND: Older patients are underrepresented in the clinical trials that determine the standards of care for oncological treatment. We conducted a review to identify whether there have been age-restrictive inclusion criteria in clinical trials over the last twenty five years, focusing on patients with metastatic gastroesophageal cancer. METHODS: A search strategy was developed encompassing Embase, PubMed and The Cochrane Library databases. Completed phase III randomised controlled trials evaluating systemic anti-cancer therapies in metastatic gastroesophageal malignancies from 1st January 1995 to 18th November 2020 were identified. These were screened for eligibility using reference management software (Covidence; Veritas Health Innovation Ltd). Data including age inclusion/exclusion criteria and median age of participants were recorded. The percentage of patients ≥ 65 enrolled was collected where available. The change over time in the proportion of studies using an upper age exclusion was estimated using a linear probability model. RESULTS: Three hundred sixty-three phase III studies were identified and screened, with 66 trials remaining for final analysis. The majority of trials were Asian (48%; n = 32) and predominantly evaluated gastric malignancies, (86%; n = 56). The median age of participants was 62 (range 18-94). Thirty-two percent (n = 21) of studies specified an upper age limit for inclusion and over half of these were Asian studies. The median age of exclusion was 75 (range 65-80). All studies prior to 2003 used an upper age exclusion (n = 12); whereas only 9 that started in 2003 or later did (17%). Among later studies, there was a very modest downward yearly-trend in the proportion of studies using an upper age exclusion (-0.02 per year; 95%CI -0.05 to 0.01; p = 0.31). Fifty-two percent (n = 34) of studies specified the proportion of their study population who were ≥ 65 years. Older patients represented only 36% of the trial populations in these studies (range 7-60%). CONCLUSIONS: Recent years have seen improvements in clinical trial protocols, with many no longer specifying restrictive age criteria. Reasons for poor representation of older patients are complex and ongoing efforts are needed to broaden eligibility criteria and prioritise the inclusion of older adults in clinical trials.

Managing Human Subjects Research During a Global Pandemic at an Academic Center: Lessons Learned From COVID-19.

By March 2020, New York City became the early epicenter of the COVID-19 pandemic in the United States. Consequently, Columbia University, with its large portfolio of human subjects research, had to address the challenges of protecting thousands of research participants and research staff from potential exposure to COVID-19 while facilitating essential biomedical research, especially pandemic-related studies. The authors describe, from the perspective of Columbia's research administration leadership, how the University and its primary teaching hospital ramped down-and later ramped up-human subjects research and reflect on lessons learned. As the pandemic unfolded, only studies offering the prospect of direct benefit to subjects were permitted to continue with in-person contact. New in-person participant enrollment ceased, except for COVID-19 prevention or treatment studies. Centralized, frequently updated communication about policies and procedures was disseminated to the Columbia research community. Procedural efficiencies were rapidly developed and implemented for review and oversight of human subjects research and contractual agreements for clinical trials. More frequent institutional review board meetings and 24-hour support markedly reduced turnaround time for COVID-19 studies, without delaying approvals of non-COVID-19 research. Research administration worked closely with relevant principal federal agencies, whose regulatory flexibility facilitated the efficient implementation of COVID-19-related research. Overall, the ramp-down and ramp-up of the majority of human subjects research, with specified priorities and accelerated processes, worked well. Adjustments were made to handle the increase in administrative tasks, the need to respond rapidly to added oversight responsibilities, and the management of the many new COVID-19-related research protocols. Timely, centralized communication, support for staff needs, prioritization, and collaboration were critical to successful research oversight at a large-scale academic setting in the midst of a pandemic. These perspectives may be useful to academic research centers addressing the current and future pandemics.

The Impact of Coronavirus Disease 2019 (COVID-19) on Retinal Microcirculation in Human Subjects.

BACKGROUND: The aim of the study was to assess retinal microcirculation in patients with coronavirus disease 2019 (COVID-19) through the use of optical coherence tomography angiography (OCT-A) and compare the results with those obtained in healthy controls. METHODS: The study enrolled 39 patients who had fully recovered from COVID-19 and 40 healthy controls. OCT-A image acquisitions were obtained using AngioVue software (version 2017.1.0.151) and the RTVue XR Avanti imaging system (Optovue Inc., Fremont, CA, USA). Nonflow area in the superficial capillary plexus (SCP), foveal avascular zone (FAZ) area in the whole retinal vasculature, FAZ perimeter, acircularity index of FAZ, and foveal density were automatically obtained with the FAZ assessment tool. Vessel density (VD) at the SCP and deep capillary plexus were also measured. RESULTS: Compared to the control group, the nonflow area and the FAZ area in the whole retina was greater in the COVID-19 group; however no statistically significant difference was observed (p > 0.05 respectively). As for vessel densities, all superficial parafoveal VD parameters were considerably higher in the COVID-19 group compared to the control group (p < 0.05 respectively). Despite the fact that the vessel densities in the remaining zones were lower in the COVID-19 group, those differences were not statistically significant (p > 0.05 respectively). CONCLUSION: VD at the parafoveal area of the SCP was significantly higher among patients in the late post-recovery period of COVID-19 disease compared to healthy controls. These findings show the impact of COVID-19 on the retinal microvasculature and its possible role as a risk factor for the development of ocular diseases.

From Consultant to Co-investigator: One Diabetes Research Team's Journey of Patient Engagement.

Patients with diabetes are experts in the lived experience of self-management, making patient engagement beyond the role of research "subject" imperative for the creation of health care solutions that meaningfully address the problems they identify. We discuss our research team's relationship with our university's College Diabetes Network (CDN), an advocacy and support group for emerging adults with diabetes. Our collaborative research relationship has spanned three years, and multiple research studies with members serving as co-designers, consultants, and co-investigators. We discuss the CDN's role in two particular studies in which CDN members made substantive contributions to study design, instrument adaptation, and interpretation of findings. Key CDN members played a larger role in research activities and facilitated sustained engagement with the larger university CDN chapter. Barriers encountered included navigating research regulatory requirements while engaging CDN members in research and facilitating sustained engagement as CDN membership changes.

Willingness of Women with Endometriosis Planning to Undergo IVF to Participate in a Randomized Clinical Trial and the Effects of the COVID-19 Pandemic on Potential Participation.

The Pre-IVF Treatment with a GnRH Antagonist in Women with Endometriosis (PREGnant) Trial (clinicaltrials.gov no. NCT04173169) was designed to test the hypothesis that 60-day pre-treatment with an oral GnRH antagonist in women with documented endometriosis and planning an IVF cycle will result in a superior live birth rate to placebo. Eight hundred fourteen women are required from 4 national sites. To determine the feasibility of using an electronic medical record (EMR)-based strategy to recruit 204 participants at the Colorado site, we conducted a survey of women within the UCHealth system. Eligible women, identified using relevant ICD-10 codes, were invited to complete a 6-question survey to assess planned utilization of IVF, potential interest in participation, and whether delays in treatment due to COVID-19 would influence their decision to participate. Of 6354 age-eligible women with an endometriosis diagnosis, 421 had a concurrent infertility diagnosis. After eliminating duplicates, 212 were emailed a survey; 76 (36%) responded, 6 of whom reported no endometriosis diagnosis. Of the remaining 70, 29 (41%) were planning fertility treatment; only 19 planned IVF. All 19 expressed interest in participation. COVID-19 delays in treatment were not considered as a factor affecting participation by 8/19; the remaining 11 felt that it would "somewhat" affect their decision. None reported that they would not consider participation because of COVID-19. EMR-based recruitment for an endometriosis clinical trial is feasible although the overall yield of participants is low. Delays in treatment due to COVID-19 did not appear to overly influence potential recruitment.

Challenges of Clinical Research Administration During the COVID-19 Pandemic.

In early 2020, clinicians and researchers rushed to understand the SARS-CoV-2 virus and how to go about treating and preventing it. Caring for patients while simultaneously learning about a disease not seen before created challenges on several levels. Much of the spotlight was on the researchers doing this critical work; however, these narratives remind us of the enormous effort and commitment shown by IRB members and research administrators responsible for research infrastructure. Despite the sense of urgency and obligation to plan and conduct clinical research during the pandemic, IRBs guaranteed that researchers still adhered to the core ethical principles that protect the rights and welfare of human subjects so that critical research could continue. Many themes emerge in these stories, including the need for flexibility in processes for both staff and research participants and the perception that IRB members serve as "research gatekeepers." With approaches to clinical research evolving, the SARS-CoV-2 pandemic may be the catalyst needed to make sustainable improvements to our research processes, roles, and goals.

Protecting Research Subjects During a Pandemic.

This commentary discusses twelve stories in which people who are involved in institutional review board (IRB) administration or serve as IRB members tell the stories of how the COVID-19 pandemic affected their work and lives. Among the aspects of these stories it highlights are the need to focus on the well-being of the institution's employees, and how issues involving protecting vulnerable subjects might relate to current policy debates about underserved communities. The final portion of this commentary focuses in particular on how one might measure success for a program in protecting its research subjects during a pandemic.

Cancer Clinical Trial Participation at the 1-Year Anniversary of the Outbreak of the COVID-19 Pandemic.

Importance: During the initial outbreak of the COVID-19 pandemic, cancer clinical trial participation decreased precipitously. Given the continued pandemic-especially the severe wave of new cases and deaths in winter 2020 to 2021-a vital question is whether trial enrollments have remained low or even worsened. Objective: To examine the experience of cancer clinical trial enrollment 1 year after the COVID-19 outbreak. Design, Setting, and Participants: This cohort study examines initial enrollments to treatment trials and cancer control and prevention (CCP) trials conducted by the SWOG Cancer Research Network between January 1, 2016, and February 28, 2021. Participants include patients enrolled in the trials. Exposures: Landmark time points reflecting the onset and the apex, respectively, of the initial COVID-19 wave (March 1 to April 25, 2020) and the winter 2020 to 2021 wave (October 4, 2020, to January 23, 2021). Main Outcomes and Measures: This study used interrupted time-series analysis to examine enrollments over time related to the COVID-19-derived exposure variables using negative-binomial regression. Relative risk (RR) estimates representing weekly enrollment changes compared with expected rates (had the pandemic not occurred) were derived. The numbers of enrollments lost during the pandemic were estimated. Results: Overall, 29 398 patients (mean [SD] age, 60.3 [13.2] years) were enrolled (24 034 before the pandemic and 5364 during the pandemic), with 9198 patients (31.3%) aged 65 years or older, 17 199 female patients (58.6%), 3039 Black patients (10.8%), and 2260 Hispanic patients (7.9%). Most enrollments (19 451 [66.2%]) were to treatment trials. During the initial COVID-19 wave, there was a 9.0% model-estimated weekly reduction in enrollments (RR, 0.91; 95% CI, 0.89-0.93; P < .001), with effects compounding each week. Enrollment recovered thereafter, but decreased again during the winter 2020 to 2021 wave, although by only 2.0% each week (RR, 0.98; 95% CI, 0.97-0.99; P < .001). Overall, during the pandemic, actual enrollments were 77.3% of expected enrollments (5364 of 6913 enrollments; 95% CI, 70.5%-85.0%; P < .001). Actual enrollments were 54.0% of expected enrollments for CCP trials (1421 of 2641 enrollments; 95% CI, 43.0%-67.0%; P < .001) and 91.0% of expected enrollments for treatment trials (3922 of 4304 enrollments; 95% CI, 81.0%-102.0%; P = .12). Conclusions and Relevance: In this cohort study, clinical trial enrollments decreased during the full year of the COVID-19 pandemic. Enrollment reductions were primarily to CCP trials, whereas, remarkably, there was not strong evidence of enrollment reductions to treatment trials. This finding suggests that clinical research rapidly adapted to the circumstances of enrolling and treating patients on protocols during the COVID-19 pandemic.

Variability in expiratory trajectory angles during consonant production by one human subject and from a physical mouth model: Application to respiratory droplet emission.

The COVID-19 pandemic has highlighted the need to improve understanding of droplet transport during expiratory emissions. While historical emphasis has been placed on violent events such as coughing and sneezing, the recognition of asymptomatic and presymptomatic spread has identified the need to consider other modalities, such as speaking. Accurate prediction of infection risk produced by speaking requires knowledge of both the droplet size distributions that are produced, as well as the expiratory flow fields that transport the droplets into the surroundings. This work demonstrates that the expiratory flow field produced by consonant productions is highly unsteady, exhibiting extremely broad inter- and intra-consonant variability, with mean ejection angles varying from ≈+30° to -30°. Furthermore, implementation of a physical mouth model to quantify the expiratory flow fields for fricative pronunciation of [f] and [θ] demonstrates that flow velocities at the lips are higher than previously predicted, reaching 20-30 m/s, and that the resultant trajectories are unstable. Because both large and small droplet transport are directly influenced by the magnitude and trajectory of the expirated air stream, these findings indicate that prior investigations of the flow dynamics during speech have largely underestimated the fluid penetration distances that can be achieved for particular consonant utterances.

Exploring the impact of COVID-19 on the willingness of older adults to participate in physiology research: views from past and potential volunteers.

We explored the views of older (≥65 years) past and potential volunteers in regard to participating in physiology research during the COVID-19 pandemic. Using an online questionnaire and focus groups, we found that past volunteers (n = 55) were more likely to take part in both acute (p < 0.05) and chronic (p < 0.05) physiology studies, compared with potential future volunteers (n = 57). Both cohorts demonstrated a positive attitude towards volunteering during the COVID-19 pandemic, although concern was evident. Novelty: Volunteers demonstrated a positive attitude and also concern towards participating in physiology research during COVID-19.

Health Communication and Decision Making about Vaccine Clinical Trials during a Pandemic.

The COVID-19 pandemic has magnified the importance of clinical trials for finding a safe and effective vaccine to protect against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes COVID-19. Although communication about vaccines and vaccine hesitancy were challenges long before COVID-19, the twin facts of a pandemic and an "infodemic" of health information, misinformation, and disinformation have raised new challenges for vaccine-related communication and decision-making. The goal of this commentary is to highlight strategies to improve communication and decision-making for adults considering participation in COVID-19 vaccine clinical trials. First, I present a general conceptual model for clinical trial participation that can be applied to various vaccine and other clinical trial contexts. Next, I introduce the ASK (Assume, Seek, Know) approach for enhancing clinical trial participation: (1) assume that all patients will want to know their options, (2) seek the counsel of stakeholders, and (3) know your numbers. The ideas presented in this commentary are intended to enhance vaccine-specific clinical trial communication, decision-making, and literacy, while dually offering strategies and resources that may help reduce vaccine hesitancy and increase vaccine uptake over time.